Number of observations: 2930 Number of features: 82 Feature Selection for House Price Prediction
2025-04-12
Graphical Interpretation
Lasso Path
- The Lasso path plot shows how the coefficients of the features change as λ increases.
- As λ becomes larger, more coefficients shrink towards zero, resulting in a simpler model.
- This plot demonstrates how coefficients are penalized as λ increases, leading to the elimination of less important features.
![]()
Cross-Validation for Lambda Selection
- Cross-validation is used to select the optimal λ value.
- A cross-validation plot shows the model’s error for different values of λ, with the best λ corresponding to the point where the error is minimized.
- This plot helps balance model complexity and generalization by selecting the λ that yields the lowest error.
![]()
Results
The histogram shows the distribution of residuals (prediction errors) after cleaning, with no non-finite values. Residuals are centered around zero, indicating a good fit of the model to the data, with no significant bias.
# Calculate residuals and clean up any non-finite values
residuals_cv_clean <- na.omit(y_test - y_pred_cv)
# Plot histogram of cleaned residuals without scientific notation
ggplot(data.frame(residuals_cv_clean), aes(x = residuals_cv_clean)) +
geom_histogram(binwidth = 50000, fill = "lightblue", color = "black") +
scale_x_continuous(labels = scales::label_number()) + # Remove scientific notation on x-axis
labs(title = "Residuals Distribution (Cleaned)", x = "Residuals", y = "Frequency") +
theme_minimal()
The cross-validation curve shows the relationship between the cross-validation error and lambda (regularization parameter). The optimal value of lambda is chosen at the minimum error (lambda.min), balancing model performance and complexity.
# Plot cross-validation curve
plot(cv_lasso)
title("Cross-validation Error vs Log(Lambda)", line = 2.5)
The coefficient path plot illustrates how feature coefficients change as lambda increases. As regularization strengthens, less important features’ coefficients shrink to zero, indicating which variables are most influential in predicting house prices.
lasso_full <- glmnet(x_train, y_train, alpha = 1)
# Plot coefficient paths
plot(lasso_full, xvar = "lambda", label = TRUE)
title("LASSO Coefficient Path")
The bar plot shows the non-zero coefficients retained by the LASSO model after regularization. These are the most influential features in predicting house prices, with higher coefficients indicating greater importance.
# Get non-zero coefficients
coef_lasso <- coef(lasso_model)
non_zero_coef <- coef_lasso[coef_lasso[, 1] != 0, , drop = FALSE]
# Create a tidy dataframe from non-zero coefficients
non_zero_df <- as.data.frame(as.matrix(non_zero_coef))
non_zero_df$Feature <- rownames(non_zero_df)
colnames(non_zero_df)[1] <- "Coefficient"
non_zero_df <- non_zero_df[non_zero_df$Feature != "(Intercept)", ] # Remove intercept if present
print(non_zero_coef)
# Plot
ggplot(non_zero_df, aes(x = reorder(Feature, Coefficient), y = Coefficient)) +
geom_col(fill = "steelblue") +
coord_flip() +
labs(title = "Selected Features by LASSO", x = "Feature", y = "Coefficient") +
theme_minimal()